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SYSTEM AND METHOD FOR FORMING 
IMAGES FOR DISPLAY IN A VEHICLE 

RELATED APPLICATION 

This application claims the benefit of U.S. 

provisional application Serial No. 60/240,510, filed 

October 13, 2000, entitled System and Method for Forming 
Images for Display in a Vehicle . 

TECHNICAL FIELD OF THE INVENTION 

This invention relates in general to image display 
systems and, more particularly, to a system and method 
for forming images for display in a vehicle. 
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BACKGROUND OF THE INVENTION 

During daylight hours, the driver of a vehicle is 
able to readily detect and recognize objects that would 
be difficult or impossible to detect or recognize at 
night. For example, assume that a deer wanders into the 
road approximately 500 meters ahead of the vehicle. If 
this scenario occurs in the middle of a sunny day, the 
driver will not only be able to detect the fact that 
something is present ahead, but will readily recognize 
that it is a deer. On the other hand, if this same 
scenario occurs at night, particularly when the only 
illumination is from the headlights of the vehicle, the 
driver will not be able to detect that anything is there, 
much less recognize that it is a deer, because the deer 
will be beyond the range of the headlights. Moreover, by 
the time the driver does detect that something is in the 
road, and well before the driver can recognize what it 
is, the driver will be much closer to the deer than would 
be the case during daylight hours. Accordingly, the risk 
of a resulting accident is much higher at night than 
during the day. 

Consequently, in order to supplement the natural 
vision of a driver, and thus reduce the risk of 
accidents, night vision systems have been developed for 
vehicles, including automobiles sold in the consumer 
market. Typical night vision systems include an infrared 
camera unit, which gathers information regarding the 
scene in front of the vehicle, mounted in the grill of 
the vehicle and a head-up display, which projects an 
image derived from information provided by the camera 
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unit onto the windshield, mounted in the vehicle's 
dashboard . 

In these systems, the image projected generally has 
field of view of approximately twelve degrees horizontal 
5 by four degrees vertical. While this field of view is 
acceptable for driving on a straight, level road, the 
field of view is inadequate when, for example, 
negotiating curves, where it would be helpful to observe 
into the turn, and/or climbing hills, where it would be 

10 helpful to observe the top of the hill while cresting it. 
Unfortunately, a simple solution to this limited field 
view problem is not readily available. 

The most straightforward solution would be to 
increase the size of the image projected onto the 

15 windshield to display a larger field of view. However, 
the space in which the head-up display must fit within 
the dashboard severely limits such efforts. 

Another solution is to minify the image projected 
onto the windshield, thereby projecting more of the scene 

2 0 in front of the vehicle onto the same area of the 
windshield. Minif ication, however, decreases the size of 
the objects that the driver sees in the projected image 
relative to the actual size of the objects and, hence, 
reduces the driver's ability to perceive the range of 

2 5 objects seen in the image. 
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SUMMARY OF THE INVENTION 

The present invention provides a system and method 
that substantially reduce or eliminate at least some of 
the disadvantages of the prior art. Accordingly, the 
present invention, at least in some embodiments, provides 
a system and method that adjust the image projected to 
the driver of the vehicle based on the heading of the 
vehicle . 

In particular embodiments, a system for forming 
images for display in a vehicle includes a lens system 
directing energy from a scene toward a detector and a 
display unit coupled to the detector. The display unit 
forms an image using information received from the 
detector. The system also includes the detector, which 
includes an array of detector elements, each detector 
element receiving energy from a portion of the scene and 
converting the received energy into information 
representative of the received energy. The detector 
sends the information associated with at least some of 
the detector elements to the display unit. The system 
further includes a computer coupled to the detector. The 
computer receives heading information, selects the 
detector elements that should be used to form an image 
based on the heading information, and instructs the 
detector which detector elements should have their 
associated information sent to the display unit. 

In certain embodiments, a method for forming images 
for display in a vehicle includes directing energy from a 
scene toward a detector, receiving the energy from a 
portion of the scene at each of a plurality of detector 
elements, and converting the energy received at each 
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detector element into information representative of the 
received energy. The method also includes determining a 
heading information of the vehicle, selecting the 
detector elements that should be used to form an image 
5 based on the heading information, and forming the image 
using the information associated with the selected 
detector elements. 

The present invention possesses several technical 
features and advantages. For example, in some 

10 embodiments, the present invention allows the image 
displayed during a turn to be adjusted from directly in 
front of the vehicle to the inside of the turn, thereby 
improving the driver's view of the upcoming roadway 
during the turn. As another example, in particular 

15 embodiments, the present invention allows the image 
displayed while climbing a hill to be adjusted from 
directly in front of the vehicle to lower on the horizon, 
thereby improving the driver's view of the upcoming 
roadway while climbing, especially when cresting the 

20 hill. As a further example, in several embodiments, the 
present invention provides a camera unit that is 
relatively compact in size. Of course, particular 
embodiments may incorporate multiple and/or additional 
technical features and/or advantages . 

2 5 Other technical features and advantages will be 

readily suggested to those skilled in the art from the 
following figures, written description, and claims. 



ATTORNEY'S DOCKET NO. 
019469 . 0218 



PATENT APPLICATION 



BRIEF DESCRIPTION OF THE DRAWINGS 

For a more complete understanding of the present 
invention, especially when viewed in conjunction with the 
following written description, reference is now made to 
5 the following drawings, in which: 

FIGURE 1 a diagrammatic view of a vehicle that 
includes one embodiment of an auxiliary vision system in 
accordance with the present invent ion ; 

FIGURE 2 is a diagrammatic view of the auxiliary 
10 vision system of FIGURE 1, showing in more detail the 
internal structure of a camera unit and a display unit of 
the auxiliary vision system; 

FIGURE 3 is a diagrammatic view of the camera unit 
of FIGURES 1 and 2, showing in more detail a lens system 
15 which is a component of the camera unit; 

FIGURE 4 is a diagrammatic view of an infrared 
detector that is a component of the camera unit of 
FIGURES 1 and 2; 

FIGURE 5 is a diagrammatic view of the auxiliary 
2 0 vision system, showing in more detail a computer which is 
a component of the auxiliary vision system; 

FIGURES 6A-D illustrate some of the different 
portions of a detector, which is a part of the camera 
unit, used during different operating conditions of the 
2 5 vehicle ; 

FIGURE 7 is a diagrammatic top view showing use of 
the camera unit of FIGURE 2 in association with standard 
detection and recognition techniques; 

FIGURE 8 is a diagrammatic view showing a portion of 
30 a scene which is imaged onto a single detector element of 
a detector disposed in the camera unit of FIGURE 2; and 
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FIGURE 9 is an enlarged view of a portion of FIGURE 
6, diagrammatically showing additional detail within the 
camera unit . 
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DETAILED DESCRIPTION OF THE INVENTION 

FIGURE 1 is a diagrammatic view of a vehicle 10 
incorporating one embodiment of an auxiliary vision 
system 2 0 in accordance with the present invention. The 
5 auxiliary vision system 20 includes a camera unit 30, 
which in the illustrated embodiment is mounted at the 
front of vehicle 10, in the middle of a front grill 12. 
The camera unit 3 0 is electrically coupled at 3 9 to a 
display unit 40, which is also a part of the auxiliary 

10 vision system 20. The display unit 40 is mounted within 
an upwardly open recess provided in the top of a 
dashboard 14 of the vehicle 10, and can project an image 
onto the inside surface of a windshield 17, as indicated 
diagrammatical ly by arrow 49, for display to the driver. 

15 The display unit 40 is of a type that is commonly known 
as a head-up display (HUD) . The camera unit 3 0 is also 
electrically coupled to a computer 60 at 69. The 
computer 60 is also part of the auxiliary vision system 
20 and provides instructions to camera unit 30 based on 

2 0 heading information it receives from an angle encoder 70, 
which is coupled to a steering column 16 of vehicle 10 
and electrically coupled to computer 60 at 79, and/or an 
inclinometer 80, which is coupled to the frame of vehicle 
10 and electrically coupled to computer 60 at 89. Angle 

2 5 encoder 7 0 and inclinometer 80, which are two types of 
sensors, are also a part of auxiliary vision system 20. 
In general, any type of sensor that can provide 
information regarding the heading of vehicle 10, such as, 
for example, steering rate, inclination rate, and/or 

30 orientation, may be used in auxiliary vision system 20. 
Additionally, one, two, or even several sensors may be 
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used in different embodiments. The auxiliary vision 
system 20 of FIGURE 1 is discussed in more detail later. 

When a driver is operating a vehicle at night, the 
driver's ability to see the road ahead is substantially 
5 more limited than would be case for the same section of 
road during daylight hours. This is particularly true in 
a rural area under conditions where there is little 
moonlight, are no street lights, and are no headlights of 
other vehicles. If an animal such as a deer happens to 

10 wander into the road at a location 500 meters ahead of 
the vehicle, the driver would readily notice and 
recognize the deer during daylight hours, whereas at 
night the deer may initially be beyond the effective 
reach of the illumination from the vehicle's headlights. 

15 Moreover, even when the headlights begin to illuminate 
the deer, the driver may not initially notice the deer, 
because the deer may be a brownish color that is 
difficult to distinguish from the surrounding darkness. 
Consequently, at the point in time when the driver first 

20 realizes that there is a deer in the road, the vehicle 
will be far closer to the deer in a nighttime situation 
than would be the case during daylight hours. There are 
many other similar high risk situations, for example, 
where a pedestrian is walking along the road. 

25 A primary purpose of auxiliary vision system 20 of 

FIGURE 1 is to provide the driver of the vehicle 10 with 
information above and beyond that which the driver can 
discern at night with the naked eye. In this regard, the 
camera unit 3 0 can detect infrared information at a 

3 0 distance well beyond the effective reach of the 
headlights of the vehicle 10. In the case of a life form 
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such as an animal or a human, the heat signature of the 
life form, when presented in an infrared image derived 
from the camera unit 30, will usually have a significant 
contrast in comparison to the relatively hotter or cooler 
5 surrounding natural environment. As discussed above, 
this is not necessarily the case in a comparable 
nighttime image based on visible light. 

Thus, in addition to the visible image that is 
directly observed by the driver through the windshield 17 

10 based on headlight illumination and any other available 
light, the auxiliary vision system 20 provides a separate 
and auxiliary image, based on infrared radiation, that is 
projected onto the inside of the windshield 17. This 
auxiliary image can provide a detectable representation 

15 of lifeforms or objects ahead that are not yet visible to 
the naked eye. Further, the auxiliary image can provide 
a much more striking contrast than a visible image 
between the lifeforms or objects and the surrounding 
scene. Note that the auxiliary vision system 20 may also 

2 0 be useful during daylight hours to supplement the view of 

objects seen with natural light. 

In an auxiliary vision system, such as the auxiliary 
vision system 2 0 of FIGURE 1, it is a generally accepted 
design principle that an object in the auxiliary image 
25 should appear to the driver to have substantially the 
same size as the corresponding real-life object appears 
to have when viewed by the driver through the windshield. 
Thus, for example, if an object along the roadway ahead 
appears to the driver to be approximately one inch tall 

3 0 when viewed through the windshield, the same object 

should appear to the driver to be one inch tall when 



ATTORNEY'S DOCKET NO. 
019469 . 0218 



PATENT APPLICATION 



11 

viewed in the auxiliary image at that same point in time. 
This is known in the art as maintaining a 1:1 
magnification factor for the auxiliary display. Of 
course, the magnification factor does not have to be 
precisely 1:1, and could for example be within 2 5% higher 
or lower than 1:1. To facilitate an explanation of the 
present invention, however, it will be assumed that the 
magnification factor is approximately 1:1. 

Auxiliary vision system 20 of FIGURE 1 involves 
competing design considerations with respect to the field 
of view (FOV) that it provides. It is, of course, 
desirable to minimize the size and weight of both the 
camera unit 30 and the display unit 40. However, if the 
effective field of view (FOV) of the auxiliary vision 
system 2 0 is varied, the minimum sizes of the camera unit 
3 0 and the display unit 4 0 tend to vary inversely. In 
particular, as the effective FOV is progressively 
increased, the size of the optics in the camera unit 30, 
and thus the size of camera unit 3 0 itself, can be 
progressively decreased, but the size of the display unit 
40 must be progressively increased. Conversely, as the 
effective FOV is progressively decreased, the size of the 
display unit 40 can be decreased, but the size of the 
camera optics, and thus the size of the camera unit 3 0 
itself, must be increased. Since the sizes of the camera 
unit 3 0 and display unit 40 thus vary inversely, a 
balance must be reached for purposes of any particular 
system. 

As a practical matter, one side of this balance is 
determined by the display unit, such as display unit 40, 
due to yet another consideration. In particular, it is 
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typically desirable to project onto the windshield, such 
as windshield 17, a relatively large image. However, as 
a practical matter, the ability to achieve this is 
limited by the extent to which the dashboard, such as 
dashboard 14, has space available to accommodate a 
display unit. In particular, if the display unit is 
given the largest physical size that is reasonable for 
the dashboard area of an average automobile, and if 1:1 
magnification is maintained, as discussed above, the 
display unit will produce an image on the windshield that 
has an effective horizontal FOV of approximately ten to 
fourteen degrees, and in the disclosed embodiment the 
horizontal FOV is approximately twelve degrees. Of 
course, the invention is compatible with a FOV larger 
than fourteen degrees or smaller than ten degrees. 
However, the disclosed embodiment uses a FOV of about 
twelve degrees because this value is typical of the 
displays presently used in existing night vision systems 
in the automotive industry. 

Given the effective horizontal FOV of about twelve 
degrees in the disclosed embodiment, as established by 
criteria relating to the display unit 40, the associated 
camera unit 3 0 must offer this needed horizontal FOV with 
a suitable degree of resolution. A feature of certain 
embodiments of the present invention is the provision of 
an improved camera unit 30, which provides the needed FOV 
with a suitable level of resolution, but which has a 
substantially reduced size, weight and cost in comparison 
to preexisting camera units. 

FIGURE 2 is a diagrammatic view of the auxiliary 
vision system 20 of FIGURE 1, showing in greater detail 
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the internal structure of both the camera unit 3 0 and the 
display unit 40. More specifically, thermal radiation 
from a scene 50 enters the camera unit 3 0 and passes 
through a lens system 32 and a chopper 34 to a detector 
36. The lens system 32 directs the incoming radiation 
onto an image plane of the detector 36. 

In the disclosed embodiment, the chopper 34 is a 
rotating disk of a known type, which has one or more 
circumferentially spaced openings. As the chopper 34 is 
rotated, it periodically permits and prevents the travel 
of incoming infrared radiation to the detector 36. 

Also in the disclosed embodiment, the detector 36 is 
a commercially available focal plane array or staring 
array detector, which has a two-dimensional matrix of 
detector elements, where each detector element produces a 
respective pixel of a resulting image. In particular, 
detector 3 6 is an uncooled pyroelectric barium strontium 
titanate (BST) detector, although numerous other types of 
detectors would also be useful in auxiliary vision system 
20 . 

The circuitry 38 is provided to control the detector 
3 6 and read out the images that it detects, and also to 
synchronize the chopper 34 to operation of the detector 
36. Further, based on information from computer 60, the 
circuitry 3 8 sends the information obtained from detector 
3 6 through the electrical coupling 3 9 to the circuitry 42 
within the display unit 40. 

The circuitry 42 controls a liquid crystal display 
(LCD) 44, which in the disclosed embodiment has a two- 
dimensional array of 384 by 220 pixel elements. The 
display unit 40 has a horizontal to vertical aspect ratio 
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of 3, as a result of which the portion of the LCD 44 that 
is actually used is 384 by 107 pixel elements. The 
circuitry 42 takes successive images obtained from the 
detector 36 through circuitry 38, and presents these on 
the LCD 44. In the disclosed embodiment, the LCD 44 
includes backlighting that makes the image on LCD 44 
visible at night . 

This visible image is projected onto a mirror 48 
that reflects the image so as to be directed onto the 
inner surface of the windshield 17, creating a virtual 
image for the driver. Although the mirror 4 8 is shown 
diagrammatically in FIGURE 2 as a planar component, it 
actually has a relatively complex curvature that is known 
in the art, in order to compensate for factors such as 
the non-linear curvature of the windshield 17, parallax 
due to the inclination of the windshield 17, and so 
forth. The curvature also gives mirror 4 8 some optical 
power, so that it imparts a degree of magnification to 
the image that it projects onto the windshield 17. The 
mirror 48 is movably supported, and its position at any 
given time is determined by a drive mechanism 46. Using 
the drive mechanism 46, the driver may adjust the mirror 
48 so that the image on the windshield 17 is in a viewing 
position comfortable for that particular driver. This is 
analogous to the manner in which a driver may adjust a 
sideview mirror of a vehicle to a suitable and 
comfortable position. Once the driver has finished 
adjusting the mirror 48 to a suitable position, it 
remains in that position during normal operation of the 
auxiliary vision system 20. 
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FIGURE 3 is a diagrammatic view of the camera unit 
30, and shows in more detail the optical structure within 
the lens system 32. More specifically, the lens system 
32 includes an objective lens 92, which directs incoming 
radiation in a manner forming an image at an image plane 
100 that is adjacent the detector 36. The lens 92 has a 
diffractive surface pattern 94 on the side thereof 
nearest the detector 36. As discussed in more detail 
below, the disclosed embodiment uses only a subset of the 
pixel elements of the detector 3 6 for each image, and 
FIGURE 3 thus shows the radiation being imaged onto only 
a portion of the detector 36, rather than the entire 
detector 36. In reality, however, the lens system 32 is 
directing radiation from scene 50 onto all of detector 
36 . 

In the disclosed embodiment, the lens 92 is made of 
Chalcogenide glass of a type that is commonly known in 
the art as TI-1173, and which is available from Raytheon 
Company of Lexington, Massachusetts. Further, the 

objective lens 92 has a diameter of 1.5", an effective 
focal length (EFL) of 1.5", and an optical speed of f/l, 
where optical speed is a standardized measure of 
refractive strength per dimensional unit. Thus, the lens 
system 32 has a smaller aperture than preexisting 
systems. It is a feature of at least one embodiment of 
the auxiliary vision system 2 0 according to the invention 
that the camera unit 3 0 can have an EFL less than about 
2.25", and preferably less than about 1.9", while 
providing a horizontal FOV less than about fourteen 
degrees and preferably about twelve degrees for an image. 
A further feature of this embodiment is the use for lens 
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92 of an f/l optical speed, rather than the f/1.2 optical 
speed used in an existing system, because the f/l optical 
speed provides an increase of over 4 0% in the infrared 
energy imaged onto the detector 36, in comparison to use 
of an f/1.2 optical speed. When the lens 92 has an f/l 
optical speed, the resulting image information from the 
detector 3 6 has more contrast than it would if the lens 
92 had an f/1.2 optical speed. 

After passing through the lens 92, the converging 
radiation passes through an aperture stop 96, and then 
through a substantially flat diffractive field lens 98, 
which is made of a polymer material, and which has a 
diffractive surface pattern on one side. The diffractive 
surface patterns on the lenses 92 and 98 facilitate color 
correction. The lens system 32 is of a type disclosed in 
U.S. Patent No. 5,973,827. The specific prescription 
information for the lens system 32 of the disclosed 
embodiment is set forth in Table 1. 
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TABLE 1 - PRESCRIPTION FOR LENS SYSTEM 32 



Parameter 


Lens 92 


Lens 98 


Radii : 




Rl 


1 . 02740" 


Flat 






R2 


1 . 4 0 648" 


Flat 


Aspheric Coefficients: 


A4 


Rl 


0 


0 




A6 


Rl 


0 


0 




A8 


Rl 


0 


0 




A10 


Rl 


0 


0 




A4 


R2 


0 . 0342740 


0 




A6 


R2 


0 . 0110210 


0 




A8 


R2 


-0 . 0013682 


0 




A10 


R2 


0 . 0048045 


0 


Diffractive Coefficients: 


CI 


Rl 




o 




C2 


Rl 


0 


0 




C3 


Rl 


0 


0 




CI 


R2 


0 . 03 024 9 






C2 


R2 


0 


-1 . 04960 




C3 


R2 


0 


-2 .22110 


Thickness : 


0 . 32 0" 


0 .002" 


Material : 






Chalcogenide 


Polymer 








Glass 1173 




Refractive Index: 


2 . 6 


1 . 5 



FIGURE 4 is a diagrammatic view of the detector 36. 
As discussed above, the detector 3 6 includes a two- 
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dimensional array of detector elements that each 
correspond to a respective pixel of a detected image. In 
more detail, and as indicated in FIGURE 4, the detector 
36 is an array of 320 detector elements by 240 detector 
elements. Each detector element has a size of 

approximately 48.5 microns by 48.5 microns. As mentioned 
above, the disclosed embodiment effectively uses only a 
portion of the detector elements of the detector 36 at a 
time, as indicated diagrammatically by a broken line 37. 
The portion 37 of the detector 36 represents 25% of the 
detector elements, or in other words an array of 160 by 
120 detector elements. The portion 37 shown in FIGURE 4 
happens to be located in the center the detector 36, 
which is the portion used for straight, level travel of 
vehicle 10, to be discussed below, but it could be any 
portion of the detector 36. For example, it might be a 
portion of equal size that is located in the upper left 
corner of the detector 36. The camera unit 30 processes 
and outputs the information obtained from all of the 160 
by 120 detector elements located in the portion 37 of the 
detector 36. However, in the disclosed embodiment, the 
particular display unit 40 discards some of this 
information. More specifically, and as discussed above, 
the display unit 40 has a horizontal to vertical aspect 
ratio of approximately 3 . Consequently, while the 

display unit 40 uses a full 160 pixels of information 
from detector 3 6 in the horizontal direction, it uses 
only 53 pixels in the vertical direction, as indicated 
diagrammatically by dotted lines in FIGURE 4. A further 
consideration is that, since the circuitry 38 receives 
images with fewer pixels, the processing load is reduced, 
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and thus less circuitry and/or slower circuitry can be 
used, which reduces the cost of the circuitry 38. 

Note that detector 3 6 uses a smaller number of 
detector elements to detect an image in the scene 5 0 than 
display unit 4 0 has pixels to form the image, 16 0 x 12 0 
v. 384 x 22 0. Thus, to avoid minification of the image, 
the information from the selected detector elements must 
be distributed over all of the pixels. To accomplish 
this, an interpolation scheme, such as, for example, the 
well known nearest neighbor algorithm, is used to 
interpolate between the information determined by each 
detector element used to detect the image to produce 
additional information for display unit 40. In 
particular, the interpolation scheme is used to generate 
information for each pixel of the display unit 4 0 that 
does not have a corresponding detector element. Of 
course, in other embodiments, detector 3 6 could use the 
same or a greater number of detector elements to detect 
an image than display unit 40 has pixels. 

FIGURE 5 is a diagrammatic view of auxiliary vision 
system 20, showing in more detail the internal structure 
of computer 60. As illustrated, computer 60 includes a 
processor 62 coupled to a memory 64, an interface 66, and 
an interface 68. Memory 64 includes a program 65 that 
has instructions for selecting the detector elements to 
be used to form an image based on heading information. 

In operation, processor 62 receives heading 
information over electrical coupling 79 from angle 
encoder 70 and over electrical coupling 89 from 
inclinometer 80 through interface 66. Using this heading 
information, processor 62, according to the instructions 



ATTORNEY'S DOCKET NO. 
019469 . 0218 



PATENT APPLICATION 



20 

in program 65, selects the detector elements of detector 
3 6 for which the associated information should be used to 
form an image by display unit 40. Processor 62 sends 
appropriate instructions regarding these detector 
elements to circuitry 3 8 of camera unit 3 0 over 
electrical coupling 69 through interface 68. Circuitry 
3 8 receives the instructions regarding the selected 
detector elements and sends the information associated 
with each selected detector element, along with any 
interpolated information, to display unit 40. 

In the illustrated embodiment, circuitry 38 obtains 
the information from the selected detector element by 
fast clocking through the unselected detector elements, 
which basically ignores the information in the detector 
elements. In other embodiments, circuitry 38 obtains the 
information from the selected detector elements by 
storing the information from all detector elements in a 
memory and then selecting the information for the 
selected detector elements. In still other embodiments, 
circuitry 3 8 dynamically alters the initial detector 
element at which to begin a scan of the detector elements 
for an image to obtain the information from the selected 
detector elements. Various other techniques exist for 
acquiring the information from the selected detector 
elements . 

Processor 62 may be an application specific 
integrated circuit (ASIC) , a digital signal processor 
(DSP) , a field programmable gate array (FPGA) , a complex 
instruction set computer (CISC) , a reduced instruction 
set computer (RISC) , or any other type of device for 
electronically manipulating electronic information. 
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Memory 64 may be random access memory (RAM) , compact-disk 
read-only memory (CD-ROM) , registers, or any other type 
of optical or magnetic volatile or non-volatile 
information storage device. In particular embodiments, 
5 computer 60 is coupled to the sensors and the camera unit 
30 through a control area network (CAN) bus. 

In certain embodiments, computer 60 may include 
image processing software, which could be part of program 
65 or a separate program, to determine the direction of 

10 the roadway, which can be used to determine the heading 
of vehicle 10 and, hence, select the image to display to 
the driver. To accomplish this, computer 60 would 
receive information associated with some or all of the 
detector elements, as either an analog or digital signal, 

15 and use this information as input to the image processing 
software. From this, computer 60 would, using the image 
processing software, determine the direction of the 
roadway - curvature, inclination, etc. - and use this 
determination to select the detector elements to be used 

2 0 to form an image. The image processing software could 

utilize road geometry analysis, lane analysis, or any 
other image processing technique for determining the 
current or future direction of the roadway. Note, the 
image processing software could work in conjunction with 
25 or to the exclusion of any sensor, such as angle encoder 
70 and inclinometer 80. Moreover, in other embodiments, 
the image processing software may be a part of camera 
unit 3 0 or other components of auxiliary vision system 
20 . 

3 0 FIGURES 6A-D illustrate some of the different 

portions of detector 36 used during different operating 
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conditions of vehicle 10. In FIGURE 6A, vehicle 10 is 
traveling on a straight, level road. Accordingly, angle 
encoder 70 is indicating that no turn is being performed, 
and inclinometer 80 is indicating that no inclination is 
being traversed. Thus, based on this heading 

information, computer 60 is instructing camera unit 30 to 
send the information associated with the center 160 x 120 
detector elements, along with any interpolated 
information, to display unit 40. 

In FIGURE 6B, vehicle 10 is making a hard right 
turn. Accordingly, angle encoder 70 is indicating that 
an intensive right turn is being performed, and 
inclinometer is indicating that no inclination is being 
traversed. Thus, based on this heading information, 
computer 60 is instructing camera unit 3 0 to send the 
information associated with the vertically centered, 
right most 160 x 120 detector elements, along with any 
interpolated information, to the display unit 40. 

In FIGURE 6C, vehicle 10 is traveling up a steep 
incline. Accordingly, angle encoder 70 is indicating 
that no turn is being performed, and inclinometer 80 is 
indicating that an incline is being traversed. Thus, 
based on this heading information, computer 60 is 
instructing camera unit 3 0 to send the information 
associated with the bottom most, horizontally centered 
160 x 120 detector elements, along with any interpolated 
information, to display unit 40. 

Finally, in FIGURE 6D, vehicle 10 is making a hard 
left turn while traveling down a steep incline. 
Accordingly, angle encoder 7 0 is indicating a hard left 
turn is being performed, and inclinometer 8 0 is 
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indicating that a decline is being traversed. Thus, 
based on this heading information, computer 60 is 
instructing camera unit 3 0 to send the information 
associated with the upper most, left most 160 x 120 
detector elements, along with any interpolated 
information, to the display unit 40. 

As can be seen, with the illustrated detector 36, 
the imaged portion of the scene 5 0 displayed to the 
driver of vehicle 10 may be adjusted up to six degrees to 
the left or right and up to four and one -half degrees up 
or down by adjusting portion 37 of detector 36. The 
imaged portion of the scene 50 displayed may be gradually 
adjusted from the center to the edges as the vehicle 10 
undergoes various heading changes because a new image is 
determined every sixtieth of a second, i.e., the detector 
36 is scanned at a rate of 60 Hz. In other embodiments, 
different scan rates may be used. Additionally, detector 
36 allows for eighty gradations when moving horizontally 
in one direction and for sixty gradations when moving 
vertically in one direction. These gradations allow the 
displayed image to be scanned across all of scene 5 0 that 
detector 3 6 detects in a relatively smooth manner. Of 
course, in embodiments where a larger or smaller number 
of detector elements are used, the degree to which the 
FOV presented to the driver of vehicle 10 may be moved 
will be larger or smaller, respectively. 

As evident from the foregoing discussion, the 
auxiliary vision system 2 0 takes advantage of the fact 
that the detector 36 is receiving energy from a scene 50 
that is larger than that which can be instantaneously 
displayed by display unit 40 without significant 
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minif ication to adjust the image presented to the driver 
of vehicle 10 as the heading of vehicle 10 changes. 
Thus, auxiliary vision system 20 electronically adjusts 
the line of sight of the image displayed to the driver 
5 based on the heading of vehicle 10. Additionally, 
particular embodiments of the present invention can be 
implemented with a variety of other lens 
system/ detector/display unit combinations as long as the 
detector detects a larger field of view than the display 

10 unit is able to instantaneously use. 

Returning to camera unit 3 0 of the disclosed 
embodiment, as evident from the foregoing discussion, 
camera unit 30 uses only 25% of the detector elements 
that are present in the detector 3 6 to form an image. 

15 This is partially because the detector 36 is an existing 
part that is readily available. The present invention 
may, as mentioned, alternatively be implemented with 
other detectors. As improvements are made in the 
technology for fabricating detectors, it will soon become 

2 0 possible to fabricate a detector in which the detector 

elements are smaller, for example on the order of 25 
microns by 25 microns. In such a detector, a 300 by 400 
array of detector elements would have a size which is 
only 25% of the size of an equivalent detector made using 
25 current fabrication technologies. With reference to 
FIGURE 3, this will allow the diameter of lens 92, the 
EFL of lens 92, the diameter of aperture stop 96, and the 
diameter of lens 98 to each be reduced by a factor of 
approximately 2, which in three dimensions will reduce 

3 0 the overall volume of the lens system 32 by a factor of 

approximately 8. This represents a further reduction in 
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present invention. 

One feature of certain embodiments of the present 
invention is the recognition that the size, weight and 
cost of a camera unit for an auxiliary vision system can 
be substantially reduced in comparison to preexisting 
camera units, without any change to the FOV, 
magnification, or effective level of resolution provided 
by the display unit 40. According to the invention, this 
can be achieved by using an array of detector elements in 
the camera unit 3 0 that has a substantially smaller 
number of detector elements than has been thought 
necessary in preexisting camera units. In this regard, 
and assuming for purposes of discussion that the display 
unit 40 has an ideal ability to reproduce the resolution 
of an image received from the camera unit, there are 
limits on the ability of an infrared detection system and 
its driver to detect an object which is far away, or to 
recognize details of such an object. Further, the 
ability to detect an object and recognize detail may 
vary, depending on a variety of factors. As a result, 
there are industry standards which are used to measure 
the capability of an infrared detection system and its 
driver to detect an object at a distance, and the 
capability to recognize detail regarding the object. 

One such industry standard involves the 
determination of a minimum resolvable temperature (MRT) 
and a minimum detectable temperature (MDT) , which each 
represent an average for several observers under a given 
set of ambient conditions. Since they include assessment 
by human observers of an image from an infrared detection 
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system, when they are applied in the context of a 
auxiliary vision system, they take into account the 
limitations of the eye of a driver to resolve detail. 
These standards are discussed with reference to FIGURE 7, 
which is a diagrammatic top view of the camera unit 30, a 
standard target 106 used to measure recognition 
capability based on the MRT standard, and a standard 
target 107 used to measure detection capability based on 
the MDT standard. In practice, the targets 106 and 107 
would actually each be oriented to extend perpendicular 
to the plane of FIGURE 7, so that each would face the 
camera unit 30. However, since FIGURE 7 is a 

diagrammatic view, they are oriented to be parallel to 
the plane of FIGURE 7 so they are clearly visible for 
purposes of the explanation which follows. 

More specifically, the target 106 is square, and 
includes four black bars and three white bars of equal 
width which are parallel and interleaved. If the 

distance 111 to the target 106 is not too large, an 
operator of the system containing the camera unit 3 0 will 
be able to not only detect that the target 106 is 
present, but to readily distinguish the vertical bars 
from each other. If the target 10 6 is then moved 
progressively away from the viewpoint 99, so as to 
increase the distance 111, it will eventually reach a 
position where it is possible to detect the fact that the 
target 106 is present, but not to recognize details such 
as the presence of the black and white bars. This 
position represents the limit of recognition capability, 
as opposed to detection capability. 
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If the target 106 is then moved progressively 
further away from the viewpoint 99, there will come a 
point at which the camera unit 3 0 can no longer detect 
that the target 106 is even present. In order to measure 
this detection capability, as opposed to recognition 
capability, it is common in the industry to use a 
different type of target, such as that shown at 107. The 
target 107 has the same size as the target 106 and is 
also a square target, but has a square black outer 
border, and a square white center. As the target 10 7 is 
moved progressively further from the viewpoint 99, so as 
to increase the distance 112, there will come a point at 
which it is no longer possible for the operator to detect 
the presence of the target 107. According to the MDT 
industry standard, this is the limit of detection 
capability. 

For purposes of the MRT and MDT standards, the 
position and size of a target relative to a viewpoint are 
not expressed in terms of the actual physical distance to 
the target, but in terms of the angle subtended by the 
target or a portion of it. For example, in FIGURE 7, the 
MDT target 107 subtends an angle 116. The target 106 is 
of equal size, but is closer to the viewpoint 99, and 
therefore subtends a larger angle 117. In the specific 
case of the MRT target 106, the position of the target 
106 is normally expressed in terms of cycles per 
milliradian, where one cycle 118 is the combined width of 
one black bar and one white bar. The target 106 thus has 
a cumulative width of 3.5 cycles. If the angle 117 is X 
milliradians, then in FIGURE 7 the position of the target 
106 would be expressed as 3 . 5/X cycles per milliradian. 
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Assume that the MRT target 10 6 in FIGURE 7 is at a 
position where, for purposes of the ambient conditions 
typical for night driving, it is at the maximum distance 
from the viewpoint 99 for which recognition is possible 
according to the MRT standard. In other words, if the 
target 106 was moved further away from the viewpoint 99, 
it would still be possible to detect that the target 106 
is present, but it would not be possible to recognize 
details such as the presence of the black and white bars. 
Also assume that the camera unit 3 0 is at the viewpoint 
99. In pre-existing auxiliary vision systems, it was 
assumed that each bar of the MRT target had to be imaged 
onto two or more elements of the detector in the camera 
unit (in a direction along a line extending perpendicular 
to the bars of the target) , in order to avoid a loss of 
resolution. One feature of these embodiments of the 
present invention is the recognition that the pre- 
existing approach represents overdesign, because each bar 
of the target does not need to be imaged onto two or more 
elements in order to have suitable resolution for 
purposes of achieving detection in an auxiliary vision 
system. 

More specifically, as a lower limit, it is 
sufficient if each of the black and white bars of the 
target 106 are imaged directly onto one detector element 
in a direction along a line extending perpendicular to 
the bars. This limit is referred to as the Nyquist 
spatial frequency for the camera unit 30. One feature of 
particular embodiments of the present invention is that 
the camera unit 3 0 is designed so that the Nyquist 
spatial frequency has a relationship to the limits of MRT 
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recognition which avoids use of the detector in a manner 
providing an excessive level of resolution that 
effectively increases the size, weight and cost of the 
camera unit. In this regard, these embodiments of the 
present invention provide an auxiliary vision system 
capable of providing MRT recognition of a multi-bar 
target having a Nyquist spatial frequency which is less 
than approximately 0.63 cycles per milliradian, and 
preferably less than 0.46 cycles per milliradian. In the 
disclosed embodiment, the Nyquist spatial frequency is 
approximately 0.38 cycles per milliradian. 

FIGURES 8 and 9 show a different way of expressing 
the same basic relationship. FIGURE 8 is a diagrammatic 
view of the camera unit 3 0 at the viewpoint 99, while 
observing a remote scene 120. Reference numeral 122 
designates a very small portion of the scene 12 0, which 
will be imaged onto a single detector element of the 
detector 36 within camera unit 30. The portion 122 of 
the scene 12 0 subtends an angle 12 6 with respect to the 
viewpoint 99. Stated differently, the angle 126, in 
milliradians, defines the portion of any scene which will 
be imaged onto a single detector element of the 
particular camera unit 30. 

In this regard, FIGURE 9 is an enlarged view of the 
portion of FIGURE 8 that includes the camera unit 30. 
The chopper 34 and circuitry 3 8 of the camera unit 3 0 
have been omitted in FIGURE 9 for clarity. FIGURE 9 
shows that the angle 12 6 represents the portion of any 
scene which, after optical processing by the lens system 
32, is imaged onto a single respective detector element 
131 of the detector 36. According to certain embodiments 
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of the invention, the portion of a scene imaged onto a 
single detector element has a size in the range of 
approximately 0.8 to 3.5 milliradians , and preferably in 
the range of 1.1 to 1.5 milliradians. In the disclosed 
embodiment, the camera unit 3 0 images approximately 1.3 
milliradians of the scene 50 onto each detector element 
131 . 

The present invention provides a number of technical 
advantages in certain embodiments. The physical volume 
of the lens system 32 for the camera unit 3 0 is reduced 
by a factor of approximately eight in comparison to 
preexisting night vision systems, which results in a 
substantial reduction in the overall size and weight of 
the camera unit 30. A related advantage is that the size 
of the objective lens 92 is substantially reduced. This 
reduces the amount of expensive material such as 
Chalcogenide glass that is required to fabricate the 
objective lens, and thus reduces the cost of the lens. 
Moreover, as the size of the lens is reduced, the 
effective yield from the lens manufacturing process is 
increased, which effects a further reduction in the cost 
of the lens . 

Still another reduction in size and cost is 
available from the fact that a smaller detector unit can 
be used, in particular a detector unit which has a 
substantially smaller number of detector elements than 
was used in preexisting night vision systems. This 
reduces the amount of costly infrared sensitive material 
required to make the detector unit, while increasing the 
number of detector units obtained from a given wafer for 
the approximately fixed cost of processing the wafer, and 
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also increasing the effective yield of parts from each 
wafer. Moreover, since the size of the lens system is 
substantially reduced, it becomes more practical to use 
an improved optical speed for the lens, which in turn 
increases the performance of the system. 

Taken together, these advantages provide a 
substantially reduced cost, weight and size for the 
camera unit of a auxiliary vision system, with the option 
of improved performance, all of which are important in 
highly competitive commercial markets, such as the 
consumer automotive market. In addition, reducing the 
size and weight of the camera unit permits it to be more 
efficiently and less noticeably integrated into a front 
portion of a vehicle, such as the grill or a bumper. 

Although the one embodiment has been illustrated and 
described in detail, it should be understood that various 
changes, substitutions and alterations can be made 
therein without departing from the scope of the 
invention. For example, one possible optical 

configuration for the lens system has been illustrated 
and described in detail, but it will be recognized that 
there are variations and modification to this optical 
configuration which are within the scope of the present 
invention. In addition, while the disclosed embodiment 
includes chopper 34, it will be recognized that there are 
variations of the present invention in which chopper 34 
could be omitted. 

Further, and as discussed above, the disclosed 
embodiment uses only a portion of the detector elements 
in the infrared detector, but the present invention 
encompasses use of a smaller detector having a number of 
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detector elements which more closely corresponds to the 
number of detector elements actually utilized by the 
disclosed embodiment. In addition, although the 

foregoing explanation of the disclosed embodiment sets 
forth the specific number of detector elements which are 
utilized, it will be recognized that the specific number 
of detector elements can be varied to some extent to 
accommodate various factors, including but not limited to 
a change in the effective field of view for the display 
unit. Also, although the disclosed embodiment uses a 
head-up display, it will be recognized that some other 
type of display unit could be used, for example a display 
that has a cathode ray tube (CRT) or LCD that is directly 
viewed by the driver. Other substitutions and 

alterations are also possible without departing from the 
spirit and scope of the present invention, as defined by 
the following claims. 

Additionally, although computer 60 has been 
illustrated as being a separate component of the 
invention, computer 60 could be a part of camera unit 30, 
display unit 40, or any other component of an auxiliary 
vision system. Moreover, the functions of computer 60 
could be incorporated into circuitry 38, circuitry 42, or 
any other circuitry of an auxiliary vision system. 



